Machine Learning and Sound Design

In this paper I discuss the role of Machine Learning (ML) in sound design. I focus on the modelling of a particular aspect of human intelligence which is believed to play an important role in musical creativity: the Generalisation of Perceptual Attributes (GPA). By GPA I mean the process by which a listener tries to find common sound attributes when confronted with a series of sounds. The paper introduces the basics of GPA and ML in the context of ARTIST, a prototype case study system. ARTIST (Artificial Intelligence Sound Tools) is a sound design system that works in co-operation with the user, providing useful levels of automated reasoning to render the synthesis tasks less laborious (tasks such as calculating an appropriate stream of synthesis parameters for each single sound) and to enable the user to explore alternatives when designing a certain sound. The system synthesises sounds from input requests in a relatively high-level language; for instance, using attribute-value expressions such as "normal vibrato", "high openness" and "sharp attack". ARTIST stores information about sounds as clusters of attribute-value expressions and has the ability to interpret these expressions in the lower-level terms of sound synthesis algorithms. The user may, however, be interested in producing a sound which is "unknown" to the system. In this case, the system will attempt to compute the attribute values for this yet unknown sound by making analogies with other known sounds which have similar constituents. ARTIST uses ML to infer which sound attributes should be considered to make the analogies.